Near-equilibrium measurements of non-equilibrium free energy 
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A central endeavor of thermodynamics is the measurement of free energy changes. Regrettably, 
although we can measure the free energy of a system in thermodynamic equilibrium, typically all 
we can say about the free energy of a non-equilibrium ensemble is that it is larger than that of 
the same system at equilibrium. Herein, we derive a formally exact expression for the probability 
distribution of a driven system, which involves path ensemble averages of the work over trajectories 
of the time-reversed system. From this we find a simple near-equilibrium approximation for the 
free energy in terms of an excess mean time-reversed work, which can be experimentally measured 
on real systems. With analysis and computer simulation, we demonstrate the accuracy of our 
approximations for several simple models. 
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Recent advances in nanotechnology make it increas- 
ingly possible to engineer molecular scale structures for 
the deliberate and efficient manipulation of energy, mat- 
ter and information on the nanometer scale. Artificial 
microscopic machines include heat pumps designed for 
very localized cooling; osmotic membranes built from car- 
bon nanotubes; quantum logic gates designed to manipu- 
late and stabilize quantum information; nano-structured 
thermoelectrics; devices for the capture and separation 
of carbon dioxide; and efficient photovoltaic solar cells. 

Notably, molecular scale machines typically operate far 
from thermodynamic equilibrium, limiting the applica- 
bility of equilibrium statistical mechanics. In particular, 
while at equilibrium the free energy of a system is min- 
imized (given the external constraints) and is often rel- 
atively easily measured, out of equilibrium no standard 
measurement technique exists, impeding the quantitative 
understanding of non-equilibrium behavior. To partially 
redress this deficit, we herein develop an experimentally 
tractable approach to measure the free energy of sys- 
tems away from equilibrium. We find that to a strikingly 
good approximation, the difference between the free en- 
ergy of a non-equilibrium ensemble and the equivalent 
system in equilibrium is determined by an excess mean 
time-reversed work, Eq. (13). 

We consider a physical system with a collection of 
controllable parameters A, in contact with a constant 
temperature heat bath with reciprocal temperature j3 = 
(UbT)^ 1 , where k-Q is Boltzmann's constant. The free 
energy F of the system, in or out of equilibrium, can 
be defined as (E) — S/(3, for mean energy (E) and en- 
tropy S = — J2 X P( x ) m P{ x ) m natural units. Here, x 
labels the microstates of the system. Interestingly, the 
free energy difference between any non-equilibrium en- 
semble P^ eq and the corresponding canonical equilibrium 
ensemble P^ q (x) — exp{/3 [F^ q — E\(x)]} with the same 
control parameters A is equal to the relative entropy be- 



tween the two ensembles [1]: 
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Thus, in both a thermodynamic and information theo- 
retic sense, the free energy difference between nonequi- 
librium and equilibrium ensembles measures a distance 
from equilibrium. 

A non-equilibrium ensemble is specified by a protocol A 
that describes the history of the control parameters over 
some time interval. In the corresponding time-reversed 
protocol A, the system starts with the final parameters 
Ab of the forward protocol, and then the controllable pa- 
rameters retrace the same series of changes, in reverse, 
to end with the initial values A a of the forward proto- 
col. Measurements performed on a system using a pair 
of conjugate protocols A and A are related by [2], 

<A. ; a - <^ W ) A6; a/<^ W >, 6; a • (2) 

Here, A is a measurement of the system (any real func- 
tion of the phase space trajectory), A is the correspond- 
ing time-reversed measurement, and W is the work per- 
formed on the system. The angled brackets indicate that 
measurements are averaged over an experimental proto- 
col, specified by subscripts: the first subscript indicates 
the initial preparation of the system; the second sub- 
script, after the semicolon, indicate the protocol during 
measurement. Thus "A a ; A" specifies that the system is 
equilibrated with fixed parameters A a and then the prop- 
erties of the system are measured while the system is 
driven with protocol A, whereas "Ab;A" indicates ini- 
tial equilibration at Ab followed by measurement during 
the time-reversed protocol A. If the preparation proto- 
col is not explicitly stated, as is the case in many of our 
previous papers, then implicitly the system is prepared 
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at equilibrium with the initial control parameters of the 
measurement protocol. 

We will use this expression to relate non-equilibrium 
probability distributions to cumulants of work. First, we 
replace the generic measurement A with a delta function 
of the final system configuration 5 (x — x\>). This gives 
a relation between the non-equilibrium probability of a 
configuration, and a non-linear average of the work per- 
formed on the system during the time-reversed protocol, 
starting from that configuration [2] : 

Pa(x) = (5(x- x h ) ) Aa . A 

= P^)(e-^U/(e-^) KrA . (3) 

The subscript "x; A" indicates initial preparation of the 
system in microstate x and subsequent work measure- 
ment during protocol A. Next, we rearrange the previous 
expression as in Ref. [3], 
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and expand both non-equilibrium averages as a sum of 
cumulants using the generating function, In (e° v ) — 

£~i£« (n) (i/) [4], giving 
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Recall that the first cumulant is the mean and the second 
the variance. Each term in the expansion is an 'excess' 
work cumulant, the difference between work cumulants 
of the time-reversed protocol, one when the system is 
initially prepared in a particular microstate x and one 
when initially prepared in an equilibrium-weighted en- 
semble of initial microstates. Given that the phase space 
distributions relax to the same steady state, irrespective 
of the initial microstate, it follows that each excess cu- 
mulant is finite, and converges to its asymptotic value in 
the long time limit [5]. Note that our excess mean work 
differs substantially from the excess heat of Oono and 
Paniconi [6]: ours is a time-reversed quantity, is defined 
over a finite time rather than instantaneously, and is an 
excess over a different baseline. 

These relations for non-equilibrium probabilities are 
formally exact, yet impractical. In particular, if a system 
is driven into a non-equilibrium steady state by a periodic 
perturbation, then the work performed on the system in- 
creases without bound. Both the exponential average in 



Eq. (4) and the higher-order cumulants in Eq. (6) are 
dominated by low dissipation realizations of the proto- 
col, which are extremely rare and not observable in any 
realistic experiment [7]. 

To proceed further, we require a tractable approxima- 
tion to the exact non-equilibrium probability. Truncating 
the cumulant expansion at the first term is tempting [8] , 
but in practice the higher-order cumulants cannot be en- 
tirely neglected. We instead make a weaker assumption, 
that the higher-order excess cumulants of the work are 
uninformative about the non-equilibrium probability dis- 
tributions, and therefore that the sum of higher-order ex- 
cess work cumulants is a constant, independent of initial 
microstate: 



K X ;A = K~A 



(7) 



We next develop an approximation for this con- 
stant K-^. Consider the relative entropy D(p\\q) = 
J2 x p x hi(Px/qx) between the non-equilibrium and corre- 
sponding equilibrium ensembles, under our first approx- 
imation [Eq. (7)]: 



D(P A \\P Xh ) « -/?(<W) A;X - (W) Ab . x ) + 
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Here "A; A" denotes initial preparation with control pa- 
rameters A a , enactment of protocol A, then work mea- 
surement during the conjugate protocol A. Notice that 
the mean work beginning from a given microstate x, av- 
eraged over any distribution of microstates, is equal to 
the mean work of the ensemble of trajectories beginning 
from that distribution, e.g. ((VV) X . X ) A = (VV) A . X . Simi- 
lar relations do not hold for the higher order cumulants. 

Relative entropy is not, in general, symmetric between 
two distributions, but if the distributions are similar, 
Qx = Px + dpx, the asymmetry is small. A Taylor ex- 
pansion of ]n(q x /p x ) = ln(l + dp x /p x ) about dp x /p x = 
gives, to leading order [9], 



D(p\\q) - D(q\\p) 



(px - qxf 
pI 



(10) 



Hence, we further assume that the asymmetry of the 
relative entropy is small in the near-equilibrium regime, 
£>(-Pa||-PaJ ~ D(P Xb \\PA), and therefore that the sum of 
the higher-order excess work cumulants, K^, is directly 
related to the first excess work cumulant (the excess mean 
work) by 
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Substituting this approximation into Eq. 5 gives a more 
manageable expression for the near-equilibrium probabil- 
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ity distribution, 
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Inserting Eq. (12) into the equation relating free en- 
ergy differences and relative entropy [Eq. (1)], it follows 
that the difference between the free energy of a non- 
equilibrium ensemble and the equivalent system in equi- 
librium, is approximately equal to minus one-half the ex- 
cess mean time-reversed work: 
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Here, finally, is our desired result. This free energy differ- 
ence is readily measurable since it is approximately mi- 
nus one-half the average work (W) A .^ when the system 
is prepared with protocol A starting from equilibrium at 
A a and then driven with the time-reversed protocol A, 
less the average work (VV) Ab . A when prepared in thermal 
equilibrium at Ab and then driven with the time-reversed 
protocol A. 

Let us consider a simple system for which the above 
expressions are exact. A micron-sized bead is suspended 
in water by an optical laser trap. The trap is initially at 
rest, and then moves at a constant velocity, dragging the 
bead through the fluid. This system has been studied 
experimentally [10, 11] and can be modeled by a sin- 
gle particle undergoing diffusive Langevin dynamics on a 
moving, one dimensional harmonic potential. The perti- 
nent properties of the model have been analyzed [12, 13]. 
Work distributions for a given initial particle position are 
Gaussian, with a mean work that depends linearly on the 
initial position of the particle relative to the center of the 
trap, and a position-independent variance. The third- 
and higher-order cumulants of the work are zero. Since 
the second and higher-order work cumulants are constant 
independent of the initial microstate, our first approxi- 
mation, that K „. A is independent of x [Eq. (7)], holds 
exactly. The equilibrium probability distributions are 
Gaussian, and, since the mean work is a linear function of 
position, it follows that the non-equilibrium probability 
distributions are also Gaussian, with the same variance, 
but shifted to a different mean relative to the equilibrium 
distribution. Consequently, our second approximation, 
Eq. (11), also holds exactly and our expressions for near- 
equilibrium probabilities and free energies are exact for 
this model at any driving rate. 

Empirically, our expressions are good approximations 
across a more general class of systems. To demonstrate 
this, we explore a system for which the steady state prob- 
abilities, free energies, entropies and work distributions 
can be calculated exactly (within floating point accu- 
racy). We simulate a particle diffusing over a periodic, 
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FIG. 1: (Color online) A simple driven system, amenable 
to numerical calculations, (a) Energy as a function of po- 
sition. A single particle occupies a periodic, one-dimensional 
energy landscape. The position coordinate is discretized into 
N x = 32 uniformly spaced positions per period. Energy is also 
discretized, E(x) = \N e {l + sin(2nx/N x ))/2\/N e where x is 
position and N e = 64 is the number of discrete energy bins. 
Results do not change appreciably with a finer discretization 
in either position or energy, (b) The system is initially in 
equilibrium with an external heat bath (+). At each discrete 
time step, the particle attempts to move one step left, one 
step right, or remain in the same location with equal proba- 
bilities, and the move is accepted according to the Metropolis 
criterion [16]. Every 1/v time steps, the energy surface shifts 
one position to the right. To ensure fully time-reversible dy- 
namics, we simulate l/2v time steps, shift the potential, and 
simulate another 1/2-u time steps before examining the non- 
equilibrium properties of the system. All figures are drawn 
in the rest frame of the potential. Eventually the spatial dis- 
tribution across a single periodic image converges to a non- 
equilibrium steady state (x), approximated by Eq. (12) (□). 
(c) The excess mean work (■) varies markedly as a function of 
starting position, whereas the excess work variance (o) is rel- 
atively constant, and the sum of second- and higher-order cu- 
mulants K A (•) [Eq. (6)] is even moreso, implying compen- 
sating behavior. Displayed results are for /3 = 4 and v = 1/4. 
The reciprocal temperature /3 is reported in inverse units of 
the energy difference between top and bottom of the poten- 
tial. 



sinusoidal, one-dimensional energy landscape that moves 
at a fixed velocity (Fig. 1). Phase space and energy are 
discretized, hence all interesting properties of the system 
can be efficiently calculated using dynamic programming 
algorithms [14, 15]. See Fig. 1 for simulation details. 

Figs. 2 and 3 demonstrate that for this model the 
steady state probability [Eq. (12)] and free energy 
[Eq. (13)] approximations are accurate given slowly shift- 
ing landscapes or high temperatures, and only diverge 
significantly from the exact results in strongly driven sys- 
tems. We also find that qualitatively similar results arise 
for aperiodic potentials and for different potential sur- 
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FIG. 2: (Color online) Equilibrium (+), steady state (x) and 
approximate steady state (□) [Eq. (12)] probability distribu- 
tions, for the system described in Fig. 1, at various driving 
rates and temperatures. The quality of our approximate dis- 
tributions, including overall normalization, deteriorates at low 
temperature and high driving velocity. The dotted box high- 
lights the conditions shown in Fig. 1. 
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FIG. 3: (Color online) The approximate steady state free 
energy difference per periodic image, AF approx [Eq. (13)], is 
very close to the exact steady state free energy difference 
AFexact = F\ a -A — F\ b , as shown by the fractional error 



(AF c; 



AFapproxVA-Fexact being much less than unity. 



Each curve represents a set of simulations at a given velocity 
(ranging from fast (v — 1/2, top) to slow (v = 1/64, bot- 
tom)), with reciprocal temperature f3 increasing from left to 
right. Empirically for this system, AF a pp rox is always less 
than AFexact, and the fractional error scales as (3 2 for small 
j3 (large temperatures). 



faces. 

In this paper, we have developed a practical method for 
measuring free energies in the near-equilibrium regime, 
and our simulation results indicate that the approxi- 
mate relation between free energy and excess mean time- 
reversed work is accurate a substantial distance from 



equilibrium. Our analysis should be directly applica- 
ble to existing single-molecule experiments where the re- 
verse protocol follows rapidly on the forward protocol, 
precluding equilibration [17]. We have concentrated on 
systems driven from equilibrium by a mechanical pertur- 
bation, but our relations could, in principle, be general- 
ized to other situations, for example a system driven by a 
temperature gradient [18]. Verifying our approximations 
in more complex systems will require independent mea- 
surements of non-equilibrium free energies; one possible 
approach for simple fluids would be to computationally 
estimate entropies from multi-particle distribution func- 
tions [19]. 
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